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FINAL  REPORT 


GRANT  #:  N000149710388 

PRINCIPAL  INVESTIGATOR:  Dr.  Woodrow  Barfield 
INSTITUTION;  Virginia  Tech 

GRANT  TITLE;  Advanced  Interface  Design  Using  Force  Feedback  Hand 
Controllers,  Wearable  Computers,  and  Augmented  and  Virtual  Reality- 
Displays 

AWARD  PERIOD;  1  March  1997  -  2  February  1998 

OBJECTIVE ;  The  objective  of  the  grant  was  to  purchase  SGI 
workstations,  force  feedback  hand  controllers,  head-mounted 
displays,  wearable  computers,  a  large-screen  projection  system, 
and  NT  workstations  to  support  research  in  the  design  and  use  of 
augmented  and  virtual  reality  environments,  wearable  computers, 
and  haptic  displays. 

APPROACH;  The  approach  was  twofold;  (1)  to  purchase  equipment  that 
would  be  used  to  establish  a  world-class  research  facility  to 
investigate  issues  in  the  design  and  use  of  augmented-  and  virtual 
reality  environments,  wearable  computers,  and  haptic  displays,  and 
(2)  to  perform  empirical  studies  evaluating  the  effectiveness  of 
augmented  and  virtual  reality  environments,  wearable  computers, 
and  haptic  displays. 

ACCOMPLIGHMENTE :  In  addition  to  purchasing  the  equipment  and 
establishing  the  virtual  environment  laboratory  at  Virginia  Tech, 
several  studies  using  the  equipment  have  been  run.  In  the  area  of 
wearable  computers  and  augmented  reality,  a  study  was  done  to 
examine  how  effectively  augmented  reality  (AR)  displays,  generated 
with  a  wearable  computer,  could  be  used  for  aiding  an  operator 
performing  a  manufacturing  assembly  task.  The  research 
concentrated  on  comparing  two  technologies  for  generating 
augmented  reality  displays  (opaque  vs.  see-through),  with  two 
current  types  of  assembly  instructions  (a  traditional  assembly 
instruction  manual  vs.  computer  aided  instruction).  The  study  was 
used  to  evaluate  the  effectiveness  of  the  wearable  based  augmented 
reality  compared  to  traditional*  instruction  mStho’ds,'  and  was  also 
used  to  compare  two  types  of  AR  displays  in  the  context  of  an 
assembly  task. 

FOr  the  experiment,  15  subjects  were  asked  to  assemble  a 
computer  motherboard  using  the  four  types  of  instruction;  paper 
manual,  computer  aided,  an  opaque  AR  display,  and  a  see-through  AR 
display.  The  study  was  run  as  a  within  subjects  design,  where 
subjects  were  randomly  assigned  the  order  of  instruction  media. 

For  the  AR  conditions,  the  augmented  environments  were  generated 
with  a  wearable  computer,  and  viewed  through  two  types  of 
monocular,  head-mounted  displays  (HMD) .  The  first  type  of  HMD  was 
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a  monocular  opaque  HMD,  and  the  second  was  a  monocular  see-though 
HMD.  Prior  to  the  experiment,  all  subjects  performed  a  brief 
training  session  teaching  them  how  to  insert  the  various 
components  of  the  motherboard  in  their  respective  slots.  The  time 
of  assembly  and  assembly  errors  were  measured  for  each  type  of 
media,  and  a  questionnaire  was  administered  to  each  subject  at  the 
end  of  each  condition,  and  at  the  end  of  the  e^eriment  to 
determine  the  usability  of  the  'four  instructibhhl'  media. 

The  results  of  the  experiment  indicated  that  both  augmented 
reality  conditions  were  more  effective  instructional  aids  for  the 
assembly  task  than  either  the  paper  instruction  manual  and  the 
computer  aided  instruction.  The  see-through  HMD  resulted  in  the 
fastest  assembly  times  followed  by  the  opaque  HMD,  the  computer 
aided  instruction,  and  the  paper  instructions  respectively.  In 
addition,  subjects  made  fewer  errors  using  the  AR  conditions 
compared  to  the  other  two  types  of  instructional  media.  However, 
while  the  two  AR  conditions  were  a  more  effective  instructional 
media  when  time  was  the  response  measure,  there  were  still  some 
important  usability  issues  associated  with  the  AR  technology  that 
were  not  present  in  the  non-AR  conditions.  Many  Of  the  subjects 
indicated  that  both  types  of  HMDs  were  uncomfortable,  and  over 
half  expressed  concerns  about  poor  image  contrast  with  the  see- 
through  HMDs . 

Augmented  reality  attempts  to  aid  a  user's  comprehension  of 
the  real  world  by  overlaying  meaningful  graphics  onto 
corresponding  areas  of  a  real' scene.  In  certain  appl'ifcations, 
particularly  medical  training  and  assembly,  performance  and 
understanding  may  be  facilitated  by  an  augmented  scene  perceived 
as  one  integrated  view  and  hindered  by  graphics  and  the  real  scene 
being  perceived  as  separate  parts.  Medical  images  designed  to  be 
integral  with  a  real  body  may  provide  a  better  training  setting 
than  augmented  environments  that  contain  graphics  clearly  not  part 
of  the  real  body  under  review.  In  a  maintenance  or  assembly  task, 
more  integral  images  may  increase  the  efficiency  and  effectiveness 
of  those  responsible  for  maintaining  or  assembling  electronic 
devices.  Three  studies  were  completed  to  investigate  these 
concepts  and  assess  the  impact  of  integrally  designed  augmented 
reality  environments.  The  first  study  examined  various  parameters 
involved  with  graphic  presentation  thought  to  influence  integral 
perception  of  single  graphic  augmented  scenes .  Transparency  of 
graphics,  texture,  and  specific  augmented  object  were  factorially 
investigated.  The  results  showed  that  a  semi-transparent  color 
graphic  was  perceived  as  most  integral  and  an  opaque  outline 
graphic  as  the  least  integral".  ¥he  'second  stucly'"  e’xam'ihed  the  two 
extremes  of  integral  graphics  from  experiment  one  in  augmented 
environments  where  multiple  graphics  were  represented  in  various 
augmented  scenes.  As  expected,  the  results  for  ejqjeriment  two's 
high  and  low  integral  graphics  were  perceived  as  either  integral 
or  not.  In  addition,  there  was  a  significant  lessening  of 
integral  perception  as  the  number  of  graphics  increased  from  two 
to  eight  in  the  overall  scene.  The  third  study  was  an  applied 
investigation  to  see  if  the  graphic  parameters  that  indicated  high 
integral  perception  would  yield  better  performance  over  low 
integral  graphics  for  a  varied  number  of  graphics  in  real  assembly 


tasks .  The  results  showed  significantly  faster  performance 
for  subjects  that  assembled  with  the  aid  of  highly  integral 
graphics  over  those  that  had  poorly  integrated  graphics. 

Another  completed  study  focused  on  using  haptic  displays  for 
the  visualization  of  forces  for  physics,  dynamics,  and  statics 
problems.  The  study  involved  using  a  force  feedback  joystick  to 
transmit  forces  relevant  to  statics  problems  to  students  learning 
how  forces  interact  upon  objects.  The  results  of  the  study  showed 
that  student's  \inder standing  of  the  problem  domain  was  enhanced 
using  the  force  feedback  condition. 

A  current  study  in  progress  examines  how  wearable  computers 
can  be  used  to  aid  in  performance  of  a  circuit-wiring  task  using 
either  a  wearable  computer  or  paper  equivalent  instruction.  A 
circuit-wiring  task  was  selected  because  it  is  a  generic  task  that 
is  performed  across  several  domains  such  as  airplane  assembly, 
computer,  assembly,  and  circuit  design.  In  addition,  in  contrast  to 
the  study  highlighted  above,  this  study  on  wearable  computers  will 
focus  on  the  mobile  aspects  of  computing  as  the  task  will  be 
performed  at  several  locations . 

Prior  to  the  main  experimental  task,  subjects  will  receive 
one  of  three  different  forms  of  circuit-wiring  training:  no  prior 
training,  part-task  training,  or  whole-task  training. 

Specifically,  subjects  receiving  whole-task  training  will  be 
provided  all  available  task  instructions  while  subjects  receiving 
part-task  training  will  be  provided  only  procedural  instructions. 
Following  one  of  the  training  conditions,  subjects  will  wire  a  set 
of  temporary  circuit  configurations  from  schematics  using  either  a 
voice-activated  text  and  graphics  instructional  system  run  on  the 
wearable  computer  or  a  paper-based  version  as  an  instructional 
reference. 

The  wearable  computer  used  in  this  study  will  be  a  Xybernaut 
133P  with  32  MB  RAM  running  Windows  95™.  The  wearable  computer 
will  be  outfitted  with  a  monochrome  monocular  VGA  head  mounted 
display,  noise- canceling  microphone  and  Verbex  Listen  for  Windows 
95™  speech  synthesis  software.  •  Subjects  will ube- University 
student  volunteers  with  20/20  or  corrected  to  20/20  vision  who 
will  be  unfamiliar  with  wearable  computers  and  the  task. 

Performance  will  be  accessed  by  recording  the  number  of 
errors  committed  during  circuit  wiring  and  time  to  task 
completion.  In  addition,  a  usability  analysis  will  be  performed 
evaluating  the  different  instruction  technologies.  Although  it  is 
e::q)ected  that  whole-task  training  will  produce  superior 
performance  to  part- task  and  no-task  conditions  using  both  forms 
of  instruction,  it  is  anticipated  that  the  no-task  and  part-task 
training  conditions  using  the  wearable  computers  will  be  superior 
to  the  no- task  and  part-task  conditions  using  the  paper-based 
equivalent. 

The  results  will  be  applicable  to  computer-based  training 
programs,  specifically  for  part-task  training  programs  seeking  to 
benefit  from  the  use  of  wearable  computers  relative  to  paper 
instruction.  Given  the  results  of  the  study,  we  will  be  able  to 
evaluate  the  effectiveness  of  different  instructional  technologies 
for  on-site  training  and  based  on  a  usability- lanalyais,  what 
features  of  each  technology  subjects  prefer. 


Finally,  the  first  of  a  series  of  two  studies  is  currently  in 
progress  to  investigate  the  effects  of  auditory  disturbances  on 
presence  and  performance  in  virtual  environments.  Specific  factors 
looked  at  are  the  type  of  noise  (speech  noise  and  office  noise) , 
the  volume  of  the  noise  and  the  effects  these  have  on  different 
types  of.  tasks.  As  virtual  reality  systems  become  more  prevalent 
in  the  work  place,  these  system^  will  have  to.,ppmpet,e,  for  the 
user's  attention  with  background  auditory  distractions.  The 
results  from  this  study  will  provide  valuable  information  into  how 
the  auditory  distractions  affect  performance  in  virtual 
environments  and  what  can  be  done  to  minimize  the  effects. 

CONCLUSIONS ;  We  have  shown  that  force  feedback  can  assist  subjects 
in  learning  conceptually  difficult  material.  We  have  also  shown 
that  wearable  computers  are  effective  instructional  technology  for 
maintenance  tasks .  Future  studies  will  continue  to  examine  when 
and  how  advanced  virtual  reality  technology  can  be  used  to  assist 
subjects  in  performance  of  tasks. 

SIGNIFICANCE ;  Thus  far  our  studies  have  shown  that  wearable 
computers  and  augmented  reality  technologies  are  effective  display 
media  for  several  generic  tasks,  and  specifically  are  superior  to 
computer-based  media  and  paper  instructional  media. 

PATENT  INFORMATION;  Not  applicable. 

AWARD  INFORMATION:  Dr.  Barfield  was  appointed  senior  editor  for  • 
the  virtual  environments  section  for  the  journal  "Presence: 
Teleoperators  and  Virtual  Environments",  and  was  appointed  US 
executive  editor  for  the  journal  "Virtual  Reality:  Research, 
Developments,  and  Applications". 
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